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Executive Summary 

Over the past 10 years, standardized admission tests have become an increasingly important factor in 
undergraduate admission, as a burgeoning number of applications have resulted in a more comprehensive 
approach to admission at many undergraduate institutions. At the same time, reforms in elementary and 
secondary education at both the state and federal level have elevated the importance of standardized tests 
as a tool to measure educational outcomes. Nevertheless, long-standing concerns with standardized tests 
have persisted, and the role of the ACT and SAT in determining who gains entry into the nation’s colleges and 
universities continues to be a hotly debated topic. A growing number of postsecondary institutions have adopted 
“test-optional” admission policies, and stakeholder anxieties about the role of standardized tests have never 
been higher. 

This Commission wishes to emphasize at the outset that a “one-size-fits-a 1 1 ” approach for the use of standardized 
tests in undergraduate admission does not reflect the realities facing our nation’s many and varied colleges and 
universities. These institutions differ greatly in size, selectivity and mission. At some, standardized tests are 
important predictors of students’ academic success, while at others, they add little compared to high school grades. 
Instead, the Commission focuses on five critical issues that affect all institutions, and offers recommendations 
(a summary of which is included at the conclusion of the report) related to the use of standardized tests in 
undergraduate admission and for more effective ways to serve our nation’s students. 

Regularly Question and Reassess the Foundations and Implications of Standardized Test Requirements 

• Although many colleges find benefit in using admission tests in admission decisions, it is the view of the 
Commission that there may be more colleges and universities that could make appropriate admission 
decisions without requiring standardized admission tests such as the ACT and SAT. The Commission 
encourages institutions to consider dropping the admission test requirements if it is determined that 
the predictive utility of the test or the admission policies of the institution (such as open access) support 
that decision and if the institution believes that standardized test results would not be necessary for 
other reasons such as course placement, advising, or research. 

• The Commission recommends that NACAC offer its membership the opportunity to share validity 
studies and additional research upon which college and university administrators may continually re- 
assess their own practices. Moreover, the Commission recommends that NACAC create a knowledge 
center where colleges and universities can share research in a manner that allows the association 
to serve as an unaffiliated clearinghouse for information about admission test validity. Finally, the 
Commission recommends that NACAC consider establishing a professional development opportunity 
for chief admission or enrollment officers to share best practice information about admission test 
policies and practices. Resources of this nature will allow for a discussion of research and practice at 
various types of institutions. 

Understand Test Preparation and Take Into Account Disparities Among Students With Differential Access to 
Preparation and Information About Admission Testing 

• As long as colleges and universities require standardized tests for admission, admission test preparation 
activities will thrive. The precise increase in scores as a result of test preparation for an individual is 
difficult to determine. However, all available academic research suggests that an overall point increase 
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of between 20 and 30 points on the SAT appears to be standard for test preparation activities. This 
modest gain (on the old 1600 scale) is considerably less than the 100 point or more gains that are often 
accepted as conventional wisdom. The Commission believes NACAC can help ensure that students 
and families are aware of the findings of objective research on test preparation. 

• To ensure that the Commission’s work is informed by the most current research, NACAC has 
commissioned a new, comprehensive white paper on test preparation that will be available in early 
2009. The Commission hopes that this paper will help to further research that will keep the public fully 
informed about test preparation. 

• The Commission recommends that NACAC pursue relationships with academic researchers and 
foundations that may support an extended “objective assessment” of the effects of test coaching 
methods to provide current, unbiased information to colleges and universities. The Commission calls 
on stakeholders from all sectors of the standardized testing community — including test vendors and test 
preparation companies — to commit resources necessary to establishing an ongoing research program 
at NACAC to conduct periodic research into the effects of various methods of test preparation. 

• Commission members agreed that the best form of test preparation is focused on core knowledge 
content and on skills that will help prepare students for their academic future. Members of the 
Commission agree that, due to the inevitability of test preparation, there are aspects of test preparation 
that may be considered best practice. Students, families, counselors, schools, or school districts 
seeking appropriate test preparation for standardized admission tests should require that each of the 
following attributes are present in the test preparation program: 

• Familiarity with test question format 

• Familiarity with test administration procedures 

• Alignment with skills necessary to master college preparatory coursework 

• Instruction in basic study habits and skills 

• The Commission recommends that high schools and other organizations submit research to NACAC 
with the purpose of establishing a trusted source for best practice and professional development. 
From this central knowledge base, college admission counseling professionals, supported by the 
association, can continually assess and share information about test preparation that is aligned with 
the development of student study skills, academic achievement, and test familiarization. 

Draw Attention to Possible Misuses of Admission Test Scores 

• The Commission believes that the time has come to end the practice of using “cutscores,” or minimum 
admission test scores, for merit aid eligibility. The scarcity of aid and the advantages affluent students 
have in gaining access to preparation for admission tests or pre-tests require us to demand change. 

• Accordingly, the Commission calls upon the National Merit Scholarship Corporation to cease using non- 
contextualized PSAT scores as the “initial screen" for eligibility for National Merit scholar recognition 
or scholarships. While we commend the National Merit Scholarship Corporation for providing financial 
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assistance for post-secondary education, the Commission believes that the National Merit Scholarship 
competition contributes to the misperception of test scores as sole measures of “merit" in a pervasive 
and highly visible manner. 

The Commission also asks that institutions that commit significant institutional resources towards 
funding National Merit Finalists and Semi-Finalists reconsider whether or not their commitments are 
founded upon solid and equitable principles given the current selection processes. The Commission 
also calls on the College Board to explain why it appears to condone the use of its PSAT as a cutscore, 
a practice that is clearly at odds with best practices in the use of test scores. 

The Commission believes that, as tests designed to provide information about individuals to colleges 
and universities, the SAT and ACT were never designed as measures of the quality of an institution of 
higher education. Accordingly, the Commission encourages U.S. News and World Report to eliminate 
test scores as a measure of institutional quality. The Commission believes that the continued use 
of admission test scores as college ranking criteria creates undue pressure on admission offices to 
pursue increasingly high test scores. 

The Commission takes issue with bond ratings that use standardized admission tests in their institutional 
analyses for two primary reasons. First, the Standards for Educational and Psychological Testing, as 
well as the guidelines for use of the SAT and ACT, maintain clear restrictions against using admission 
test scores for purposes other than those for which the test is designed. Neither SAT nor ACT has ever 
claimed that admission test scores are reliable indicators of institutional financial health. While there 
is undoubtedly a correlation between institutional financial resources and test scores, the Commission 
believes that the use of such scores confuses correlation with causation, and as such, comprises an 
inappropriate use of test scores. Second, the Commission believes that the continued use of admission 
test scores as bond rating criteria creates undue pressure on admission offices to pursue increasingly 
high test scores. While ratings firms may suggest that higher scores indicate more flexibility in times of 
scarcity, there is little evidence to suggest that institutions lower their test scores in such instances. 

The Commission recommends that states refrain from using standardized admission tests without 
significant modification as evaluators of student achievement, particularly when high stakes 
accountability measures are attached. Admission tests were not designed for this purpose, and are not 
sufficiently tailored to measure progress toward explicit measures for learning in a given state. 

A substantial body of research exists that can help states, school districts, and schools design 
measurements of “college readiness.” In addition, while admission tests are useful to many college and 
university admission offices, they are second to a student's performance in challenging coursework. 
Therefore, efforts to ensure that schools are preparing their students for college should not look to 
a single admission exam for an assessment of their progress, but to the strength of the school’s 
curriculum and students’ performance in those courses as their primary indicator. 

The testing agencies, including the College Board and ACT, all publish and promote guidelines for 
appropriate use of their testing products. As an additional measure, the Commission believes that a 
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periodic usage audit may be necessary and desirable to protect against misuse of these organizations’ 
testing products. There is precedent in the College Board's recently-initiated audit of Advanced 
Placement programs at high schools across the United States. The Commission believes that a similar 
audit by the College Board and ACT may help limit misuse of tests by bringing the two organizations’ 
combined weight to bear on those institutions and organizations that misuse admission test scores. 

Establish Opportunities for Colleges and Universities, As Well As College Admission Counseling Professionals, 
to Educate Themselves About the Appropriate Use of Standardized Test Scores 

• The Commission recommends that NACAC develop a fundamental training program for college 
admission officers, school counselors, and other college admission professionals to ensure best 
professional practice in the consideration and implementation of standardized admission test policies. 
This fundamental training should be provided by a source other than the testing agencies, whose 
training is most appropriate for the specific use of their own products. 

Understand Differences in Test Scores Among Different Groups of People, and Continually Assess the Use of 
Standardized Test Scores Relative to the Broader Social Goals of Higher Education 

• Researchers who study test bias focus on whether different items on the tests produce differing results 
across ethnic groups. The Commission is in agreement that the test’s questions generally measure 
what they purport to measure. A substantial body of literature indicates that test bias has been largely 
mitigated in today’s admission tests due to extensive research and development of question items on 
both the SAT and ACT. 

• Research suggests that admission test scores may be over-predictive of first year G PA for some minority 
students, and may be under-predictive of first year GPA for some female students. Research also 
suggests that differences among racial/ethnic and socio-economic subgroups persist across a wide 
range of educational indicators, including high school grades, state testing, and national tests. 

• The performance of students with limited English proficiency is a key consideration for colleges and 
universities, particularly as demographic changes in the U.S. are likely to result in greater numbers 
of English-as-a-second-language (ESL) students seeking admission to college. Predicting first-year 
grades for such students is extremely difficult. 

• The Commission understands that as practitioners, college admission counseling professionals will not 
soon settle the debate on the fairness of admission testing. However, some control clearly rests in the 
hands of postsecondary institutions to account for inequities that are reflected in test scores and other 
criteria for college admission. That choice, however, is constrained by the tendency of stakeholders 
inside and outside of the university to attach inappropriately high stakes to admission test scores. 

• The Commission is concerned that test scores appear to calcify differences based on class, race/ 
ethnicity, and parental educational attainment. To come to some resolution, the Commission agrees 
that without confusing correlation with causation, admission offices must remain aware that test score 
differences persist among already under-served populations. Part of the public mission of colleges 
and universities is to ensure that differences that are not attributable to a student’s ability to succeed 
academically at an institution are mitigated in the admission process. 
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A Future Direction for Admission Testing 

• Colleges most often determine the utility of admission test scores by assessing how predictive they 
are of first-year grades. The Commission wishes to underscore that as such, standardized admission 
tests should not be considered as sole predictors of true college success. Commission members 
unanimously agreed that college success is a term of sufficient breadth that it includes degree 
attainment, a wide range of GPAs, and the acquisition of experiences and skills that will propel a 
student into the workforce, graduate education, or responsible citizenship. For this broad definition of 
success, standardized admission tests — as well as other individual factors — are insufficient predictors 
of a student’s likelihood of overall success. 

• There are tests that, at many institutions, are both predictive of first-year and overall grades in college 
and more closely linked to the high school curriculum, including the College Board’s AP exams 
and Subject Tests as well as the International Baccalaureate examinations. What these tests have 
in common is that they are — to a much greater extent than the SAT and ACT — achievement tests, 
which measure content covered in high school courses; that there is currently very little expensive 
private test preparation associated with them, partly because high school class curricula are meant to 
prepare students for them; and that they are much less widely required by colleges than are the SAT 
and ACT. 

• Not all state high school exams are sufficient to measure the prospect of success in postsecondary 
education. However, if such tests can be developed so they predict college grades as well as or better 
than the SAT, ACT, AP, IB exams, and Subject Tests do, the Commission would urge colleges to use 
them in the admission evaluation process. 

• Such achievement tests have a number of attractive qualities. Their use in college admission sends 
a message to students that studying their course material in high school — not taking extracurricular 
test prep courses that tend to focus on test-taking skills — is the way to do well on admission tests. 
Using achievement tests would encourage high schools to broaden and improve curricula, and would 
promote a sense of transparency about what tests measure. Further alignment of college entrance 
testing and preparation with high school curricula would also reduce the inequities inherent in the 
current system of preparation for and administration of college admission tests. 

• Individually, colleges will always try to build the strongest entering classes they can, often as measured 
by test scores, but collectively they bear a larger responsibility to make the American educational 
system as good, as fair, and as socially beneficial as possible. By using the SAT and ACT as one of 
the most important admission tools, many institutions are gaining what may be a marginal ability to 
identify academic talent beyond that indicated by transcripts, recommendations, and achievement 
test scores. In contrast, the use of state level end-of-high school achievement tests, College Board 
Subject Tests and AP tests, or International Baccalaureate exams, would create a powerful incentive 
for American high schools to improve their curricula and their teaching. Colleges would lose little or 
none of the information they need to make good choices about entering classes, while benefiting 
millions of American students who do not enroll in highly selective colleges and positively affecting 
teaching and learning in America’s schools. 



NACAC Testing Commission Report, September 2008 



• A possible future direction for college admission tests is widespread adoption of tests that more fully 
reflect subject matter from high school courses. Tests administered by states may provide valuable 
information for admission purposes, particularly in public institutions that predominantly serve a given 
state’s population. However, such tests must be designed to measure a proficiency level that is more 
reflective of college readiness than minimum competency. 
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Commission Background 

Over the past 10 years, standardized admission tests have become an increasingly important factor in 
undergraduate admission, as a burgeoning number of applications have resulted in a more comprehensive 
approach to admission at many undergraduate institutions. At the same time, reforms in elementary and 
secondary education at both the state and federal level have elevated the importance of standardized tests 
as a tool to measure educational outcomes. Nevertheless, long-standing concerns about standardized tests 
have persisted, and the role of the ACT and SAT in determining who gains entry into the nation’s colleges and 
universities continues to be a hotly debated topic. A growing number of postsecondary institutions have adopted 
“test-optional” admission policies, and stakeholder anxieties about the role of standardized tests have never 
been higher. 

Purpose of Commission 

Against this backdrop, the National Association for College Admission Counseling (NACAC) appointed a 
Commission on the Use of Standardized Tests in Undergraduate Admission to make recommendations about 
standardized admission test use based on the core observations made by the National Research Council in its 
1999 Myths and Tradeoffs report. The NACAC Commission was convened by the Board of Directors to represent 
the diverse perspectives of secondary and postsecondary institutions in the United States, as well as the college 
admission counseling professionals who work to assist students with the transition to college. 

According to the Myths and Tradeoffs report, the key benefits of standardized tests include: 

■ Efficient source of comparative information for admission offices for which there is currently no 
substitute 

■ Valuable efficiencies for institutions that must review thousands of applications 

■ A high score for under-performing students may “lead admission officers to consider acceptance for a 
student who would otherwise be rejected" 

The report also articulated key drawbacks of standardized tests, particularly regarding inappropriate use of tests, 
including: 

■ Admission policies and practices not derived from and clearly linked to an institution’s overarching 
intellectual and other goals 

■ Use of test scores in admission that do not serve institutional goals 

■ Lack of clear articulation of the goals of standardized test use by colleges 

■ Use of scores as more precise and accurate measures than they are and reliance on test scores for 
making “fine distinctions among applicants” 

The purpose of the NACAC Commission’s work is to determine how NACAC, institutions of higher education, and 
other stakeholders interpret and implement policies or practices with regard to the use of standardized tests in 
undergraduate admission. The extent to which current practice reflects the conclusions made in the Myths and 
Tradeoffs report provided a framework for the Commission’s discussions. The resulting recommendations will be 
referred to the NACAC leadership and members to develop into a variety of programs and to stimulate a vigorous 
public discussion of the many critical issues surrounding standardized testing. 
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About NACAC 

NACAC is an Arlington, VA-based education association of more than 11,000 secondary school counselors, 
independent counselors, college admission and financial aid officers, enrollment managers, and organizations 
that work with students as they make the transition from high school to postsecondary education. The association, 
founded in 1937, is committed to maintaining high standards that foster ethical and social responsibility among 
those involved in the transition process, as outlined in the NACAC Statement of Principles of Good Practice. 
More information about NACAC is available at www.nacacnet.org. 
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Introduction 

A discussion of any single factor in college admission decisions is incomplete without considering the rich context 
in which such decisions exist. Providing context for college admission decisions in the 21 st century requires 
consideration of changes in postsecondary educational institutions, differentiated tools to accommodate diverse 
admission practices, the changing student population, and an assessment of the state of current admission 
practice. Each of the numerous factors that contribute to college admission decisions has its own history, 
purpose, and form. Of those, perhaps none attract as much attention as standardized admission tests. Before 
beginning a discussion of standardized tests in college admission today, the Commission believes it is important 
to consider the historical processes that have brought us to this point. 

The Evolution of Selective Admission 

Throughout the twentieth century, colleges and universities in the United States evolved in different ways. Some 
institutions have become increasingly selective, some have settled into niches that offer focused educational 
opportunities, and still others have organized themselves to serve as much of the growing population of students 
interested in postsecondary education as possible. The resulting patchwork of colleges and universities is as 
diverse as the student population it attempts to serve. The nature of admission requirements at colleges and 
universities is similarly diverse, so attempts to generalize about universal standards or processes in undergraduate 
admission are difficult. 

The admission mechanism that is most familiar to Americans today, a combination of academic and non- 
academic criteria, did not take shape until after World War I. (Wechsler, 1977) Prior to World War I, students 
applying to college typically sat for an interview and/or institutionally-administered exam at one or two colleges. 
The number of students expected to enroll in college then was drastically smaller than the number of students 
applying to college today. Between the late 1800s and early 1900s, colleges of varying types struggled to establish 
admission policies that would best serve their institutional mission. In the late 1800s, some agricultural colleges 
briefly experimented with reducing or eliminating admission standards to bolster enrollment. (Rudolph, 1990) 
These institutions later abandoned such approaches for a more basic measure of educational attainment as a 
primary admission standard in order to serve the largest number of qualified students. Around the same time, 
institutions that had provided a classical education posed the question, “why expand at a time when prestige [is] 
no longer measured by numbers, but by selectivity?" (Wechsler, 1977) Accordingly, these institutions set out to 
establish stricter admission standards to limit enrollment to students who met high academic standards. In this 
early phase of the evolution of higher education in America we find the roots of the variance among colleges and 
universities that persist to the modern era. 

As more students sought entry into postsecondary education, institutions initiated two general and distinct 
efforts to organize admission standards within and across institutions. A group of Midwestern colleges, led by 
the University of Michigan, initiated the high school certificate system, precursor of today’s modern public high 
school transcript, to establish a reliable, certified record of a student’s academic qualifications. (Wechsler, 1977) 
A group of colleges on the East coast collaborated to form the College Entrance Examination Board, precursor to 
today’s SAT admission exam and the College Board (the organization that administers the SAT). (Rudolph, 1990) 
Soon after the establishment of the College Board, the ACT (formerly American College Testing) was founded 
in the Midwest as a similar effort to standardize admission requirements in the form of a single admission test. 
(Zwick, 2007) 
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[T]he race for numbers is over, and. ..the race for quality has begun. A few 
years ago our colleges and universities were competing for students, and 
great emphasis was laid upon “healthy growth.” Now we are beginning to 
limit our numbers, to compete only for the best students, and to point with 
pride to the multitude that we now turn away. 

-- Frank Aydelotte, President, Swarthmore College (1928) 

For the present our primary task is that of taking all types of young people 
and discovering their powers. 

-- Alexander Meiklejohn, President, University of Wisconsin (1927) 
(Wechsler, 1977) 



AfterWorld War II, the selective college admission mechanism that most Americans recognize — consideration of 
high school grades, college preparatory coursework, admission test scores, recommendations, personal essays, 
extracurricular activities, and other non-academic factors in the admission decision — evolved to its present 
state. 

Standardized admission tests (SAT and ACT) are not the most important factor in college admission decisions. 
High school grades in college preparatory classes have long been the most important criterion. While grades 
are generally the most reliable predictor of first-year academic performance in college, high schools employ 
differential grading techniques, making it challenging to assess the qualifications of students from different high 
schools with different grading standards and course strength. As NACAC survey research indicates, colleges 
and universities have attributed increasing importance to standardized tests over the past decade, even while a 
large number of colleges have dropped the requirement that students submit test scores. Beyond grades and 
test scores, other factors — including many unrelated to academic achievement — complete the list of attributes 
colleges consider when reading a student’s application for admission. (See table next page) 

Colleges across the board indicate that high school grades in college preparatory courses are the most important 
factor in admission. And while standardized admission tests are consistently rated as the second most important 
factor in admission, admission officers generally agree that high grades and low test scores are preferable to the 
converse. A large body of research, some of which is conducted by testing agencies, suggests that grades and 
test scores may be used interchangeably. (Bridgeman, 2004; Camara, 2006; Hezlet, 2001; Kobrin, 2002) There 
is an equally substantive body of research, however, that suggests high school grades are better indicators of 
grades beyond the freshman year in college than admission test scores. (Geiser, 2007) 
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Percentage of colleges attributing “considerable importance” to factors in the admission 
decision: 1993 to 2006 





1993 


1994 


1995 


1996 


1997 


1998 


1999 


2000 


2001 


2002 


2003 


2004 


2005 


2006 


Grades in college prep / 
strength of curriculum 1 


82% 


83% 


80% 


78% 


81% 


79% 


84% 


78% 


80% 


76% 


78% 


80% 


74% 


- 


Grades in college prep 


” 


” 


“ 


” 


” 


” 


” 


” 


” 


” 


“ 


” 


“ 


76 


Strength of curriculum 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


62 


Admission test scores 


46 


43 


47 


48 


50 


51 


54 


58 


52 


57 


61 


60 


59 


60 


Grades in all courses 


39 


37 


41 


38 


41 


44 


42 


43 


45 


50 


54 


57 


54 


51 


Essay 


14 


17 


21 


20 


18 


19 


19 


20 


20 


19 


23 


25 


23 


28 


Class rank 


42 


40 


39 


36 


34 


32 


32 


34 


31 


35 


33 


28 


31 


23 


Counselor rec. 


22 


20 


19 


17 


20 


16 


18 


16 


17 


16 


17 


18 


17 


21 


Demonstrated interest 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


7 


7 


15 


21 


Teacher rec. 


21 


19 


18 


19 


19 


16 


14 


14 


16 


14 


18 


18 


17 


20 


Interview 


12 


12 


15 


13 


11 


11 


9 


11 


11 


10 


9 


9 


9 


10 


Extracurricular activi- 
ties/work 2 


6 


6 


7 


6 


6 


4 


5 


7 


6 


7 


7 


8 


8 


- 


Extracurricular activi- 
ties 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


8 


Work 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


3 


Subject tests (AP, IB) 


- 


- 


- 


- 


- 


- 


- 


- 


- 


6 


7 


5 


7 


8 


State exams 


- 


- 


- 


- 


- 


- 


- 


- 


- 


6 


7 


6 


7 


6 


SAT II scores 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


5 



-- Data are not available. 



1 On the 2006 survey, grades in college prep courses and strength of curriculum were listed as two separate 
factors. In previous years, one factor was listed as grades in college prep courses/strength of curriculum. 

2 On the 2006 survey, extracurricular activities and work were listed as two separate factors. In previous years, 
one factor was listed as work/extracurricular activities. 

SOURCE: NACAC Admission Trends Surveys, 1993 through 2006. 



There are limitations to what admission tests measure, a point on which the Commission is in agreement. 
A growing field of research, in education and psychology, suggests different approaches to evaluation that 
may allow for broader and more inclusive review of individual talents. The theory of multiple intelligences, 
for instance, suggests that “human cognitive competence is better described in terms of a set of abilities, 
talents, or mental skills” called “intelligences.” (Gardner, 2006) Work is also underway to develop non-cognitive 
measures for potential use by colleges and universities to help explain lingering test score differences between 
subgroups of test takers. The term “non-cognitive” refers to “variables relating to adjustment, motivation, and 
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perceptions, rather than the traditional verbal and quantitative (often called cognitive) areas typically measured 
by standardized tests.” (Sedlacek, 2004) Because the purpose of this Commission is to discuss the use of 
standardized admission tests in undergraduate admission, this report covers those tests currently in use. 

While we can generalize about the relative importance of factors in the admission decision across colleges 
and universities, (Hawkins, 2007) this Commission stresses that as a foundation for discussing the use of 
standardized admission tests for undergraduate admission, the varying form, function, and mission of colleges 
and universities prevents us from suggesting a one-size-fits-all approach. 

Today, there are 643 public four-year colleges and universities, and 1,533 private four-year, not-for-profit colleges 
and universities. (National Center for Education Statistics, 2007) The population these institutions serve is 
dramatically different from that served by colleges and universities in the past. The population of college-bound 
students will continue to change into the foreseeable future. According to the Western Interstate Commission on 
Higher Education (WICHE): 

The nation and more and more states are closing in on “majority-minority” status relative to 
public high school graduating classes, in which the number of graduates who are not White 
non-Hispanic exceeds the number of graduates who are. Between 2004-05 and 2014-15, 

WICHE projects that the nation’s public high schools will produce: 

• Almost 207, 000 more Hispanic graduates (an increase of 54 percent). 

• Nearly 46, 000 more Asian/Pacific Islander graduates (an increase of 32 percent). 

• About 12,000 more Black non-Hispanic graduates (an increase of 3 percent). 

• About 2,000 more American Indian/Alaska Native graduates (an increase of 7 percent). 

• Nearly 1 97, 000 fewer White non-Hispanic graduates (a decline of 11 percent). 

These data show that minorities account for all the growth in our public high schools' 
production of graduates. Especially noteworthy is that the projected increase in Hispanic 
graduates alone more than offsets the decrease in White non-Hispanic graduates. In fact, if 
minority students completed high school at the same rate that White non-Hispanic students 
do, this shift would be even more dramatic. 

Clearly, the composition of our schools is changing. State policymakers and officials in school 
districts, K-12 schools, and postsecondary institutions need to be aware of these changes 
and how they might impact curriculum and preparation, the demand for support services, 
the demand for postsecondary education, affordability, and other issues. (Western Interstate 
Commission on Higher Education, 2008) 

As another step in the sequence of events shaping colleges and universities in America, this demographic 
change, along with the continually expanding pool of students seeking entry into postsecondary education, is an 
important reason to carefully examine the foundations of our points of entry into higher education. 
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Challenge to Stakeholders 

The SAT and ACT have achieved nearly iconic status in America and throughout the world with powerful effects 
on public policy, social mobility, and even individual identity. (Gould, 1996) NACAC convened the Commission 
on the Use of Standardized Tests in Undergraduate Admission to determine how the association could add its 
unique perspective to the public debate about the proper role for such tests. 

The ground that can be covered in any study of standardized testing in admission is so broad that it is impossible 
to address the entire range of issues. As an illustration, the Commission received thousands of pages of materials 
from stakeholders across the nation, consisting almost entirely of summary overviews of the extensive literature 
that undergirds the discussion of testing in admission. Accordingly, the Commission has chosen to focus on 
issues that (1) NACAC is well-positioned to address, and (2) that Commission members identified as most 
pressing at this time. 

The annual ritual of the college admission process in the United States, combined with a large number of 
colleges’ continued reliance on standardized admission tests as a factor in their admission decisions, will benefit 
from continued discussion and study. For that reason, the Commission has attempted to establish in this report 
a series of recommendations that can stimulate a rich and sustained process of introspection about how our 
nation’s colleges and universities evaluate applicants for admission. 

The Commission’s challenge to colleges and universities is to assume control of the conversation about the 
use of standardized tests in admission. There are myriad ways in which admission tests are used in the United 
States, and myriad ways in which students intersect with the process of admission testing. The Commission 
found that the primary challenge in every case is to ensure that colleges and universities are able to think 
and communicate clearly, independently, and progressively about their use of standardized admission tests. 
Transparency, particularly at each individual institution, about standardized test use can emphasize the diversity 
of higher education institutions in America. A secondary, though important, challenge is to ensure that misuses 
and mischaracterizations of admission tests do not further perpetuate popular overstatement of the importance 
of these tests in the admission process. 

Accordingly, the NACAC Commission on the Use of Standardized Tests in Undergraduate Admission recommends 
the following: 

1. ) Regularly question and re-assess the foundations and implications of standardized test requirements; 

2. ) Understand test preparation and take into account disparities among students with differential access 

to information about admission testing and preparation; 

3. ) Draw attention to possible misuses of admission test scores; 

4. ) Establish opportunities for colleges and universities to educate themselves and their staff about the 

appropriate uses of standardized tests; 

5. ) Understand differences in test scores among different groups of people, and continually assess the use 

of standardized test scores relative to the broader social goals of higher education. 

The remainder of the report is organized around these five recommendations. Each recommendation will be 
expanded upon in subsequent chapters of the report. 
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1. Taking Back the Conversation 

Regularly question and re-assess the foundations and implications of standardized test 
requirements 

The national conversation about the relative importance of standardized admission tests is dominated by the 
media, commercial interests, and organizations outside of the college admission office, including rankings 
publications. Commission members frequently posed the question, “have the negatives associated with 
standardized admission testing come to outweigh the positives?” The Commission agreed that while it could not 
provide a definitive response to this question for all colleges and universities, there were some incontrovertible 
facts about admission testing in 2008. 

First, some colleges and universities enroll sufficient qualified student populations without the standardized 
admission tests. In fact, more than 280 four-year, not-for-profit institutions do not currently require the 
submission of standardized admission test scores for admission. Accordingly, standardized admission tests are 
not universally essential for colleges and universities seeking to make good admission decisions. One critic of 
test-optional admission policy posed the questions, “Why would [a] college not want to use as much information 
as it could gather in determining admissions? Why is less evidence better than more?" (Locurto, 2005) The 
question, we believe, is not of more versus less evidence. This argument begs the question of why colleges don’t 
simply require every test score available, including both SAT and ACT, College Board Subject Tests, graduation 
exams and Advanced Placement or International Baccalaureate exams. Rather, the question is about the utility 
of the means by which institutions achieve their mission-driven enrollment ends. 

Admission tests provide an efficient tool for comparison, and the strength of the correlation between test scores 
and first year college grades is generally understood. However, when combined with high school grades, the 
additional predictive utility of these tests in making admission decisions has been debated. Results vary from 
institution to institution largely as a function of the score variability among the applicant pool. It is therefore 
incumbent on each institution to determine, through institution-specific validity studies, whether the predictive 
utility of these tests warrant their use as admission tools. Although many colleges find benefit in using admission 
tests in admission decisions, the Commission believes that there may be more colleges and universities that 
could make appropriate admission decisions without requiring standardized admission tests such as the ACT 
and SAT. The Commission encourages institutions to consider dropping the admission test requirements if it is 
determined that the predictive utility of the test or the admission policies of the institution support that decision 
and if the institution believes that standardized test results would not be necessary for other reasons such as 
course placement, advising, or research. 

Second, colleges and universities must be able independently to establish, support and explain their admission 
criteria and selection processes. Whether an institution is test-optional or requires tests should be determined 
by conducting research on the utility of standardized tests to determine if the additional predictive utility of the 
tests warrants their use in admission at that institution. Institutions must exercise independence in evaluating 
and articulating their use of standardized test scores. There is also a need for an independent forum for inter- 
institutional evaluation and discussion of standardized test use in admission that can provide support for colleges 
with limited resources to devote to institutional research and evaluation. 
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Third, institutions use standardized admission tests differently. For that reason, it is critical for colleges and 
universities to articulate clearly the emphasis or lack thereof placed on such tests and the role they play in 
admission and scholarship decisions. The table below provides an overview of admission test score use by 
various types of institution. 



Percentage of colleges attributing “considerable importance” to factors in the admission 
decision by institutional characteristics: 2006 



Enrollment 



Grades in col- 
lege prep 
courses 



Strength of 
curriculum 



Admission 
test scores 



Grades in all 
courses 



Essay/ 

writing 

sample 



Class rank 




Fewer than 3,000 
students 


72.9 


57.3 


55.5 


48.0 


29.2 


16.6 


3,000 to 9,999 


90.6 


73.6 


67.9 


50.0 


30.2 


43.4 


10,000 or more 


79.5 


68.2 


81.4 


69.0 


16.3 


34.1 



Selectivity 



Accept fewer than 50 
percent of applicants 


85.7 


80.0 


57.1 


54.2 


57.1 


30.0 


50 to 70 percent 


75.2 


62.0 


63.4 


53.0 


24.0 


27.6 


71 to 85 percent 


83.5 


66.1 


63.4 


46.9 


21.7 


20.9 


More than 85 percent 


55.6 


34.4 


54.8 


51.6 


21.3 


14.5 


Yield 


Enroll fewer than 30 
percent of admitted 
students 


90.9 


78.9 


46.1 


44.6 


32.9 


22.1 


30 to 45 percent 


82.1 


67.3 


66.4 


57.2 


22.1 


21.1 


46 to 60 percent 


61.1 


40.7 


63.6 


48.2 


33.3 


30.9 


More than 60 
percent 


43.2 


30.6 


62.2 


43.2 


35.1 


21.6 
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Commission Recommendation: Conduct Institution-Specific Validity Research 

NACAC currently maintains a statement that “college admission professionals should research the degree to 
which standardized tests predict achievement in college alone and in conjunction with other credentials for all 
students, for men and women, and for members of different racial and ethnic groups.” The current Commission 
agrees with this statement, and recommends that colleges and universities regularly conduct research of this 
nature, especially institutionally-specific validity research. While support for validity research is available from 
the testing agencies, the Commission does not believe that colleges and universities should rely solely on the 
testing agencies for it. 



Commission Recommendation: Establish a Forum for Test Validity Under the Auspices 
of NACAC 

Rather, this Commission suggests that colleges and universities create a new forum for validity research under 
the auspices of NACAC. Such an independent discussion might begin to address questions the Commission 
and other stakeholders have posed about the tests. For instance, the National Center for Fair and Open Testing 
suggested that colleges “examine whether first-year grade point average is too narrow a criteria" for evaluating 
the utility of standardized admission tests, and “whether there are alternate criteria that would provide a better 
prediction of a student's ability to persist and achieve beyond the first year.”(Rooney, 1998) 

To accommodate this discussion, the Commission recommends that NACAC offer its membership the 
opportunity to share validity studies and additional research upon which college and university administrators 
may continually re-assess their own practices. Moreover, the Commission recommends that NACAC create a 
knowledge center where colleges and universities can share research in a manner that allows the association 
to serve as an unaffiliated clearinghouse for information about admission test validity. Finally, the Commission 
recommends that NACAC consider establishing a professional development opportunity for chief admission or 
enrollment officers to share best practice information about admission test policies and practices. We encourage 
college faculty and institutional research professionals to be a part of this discussion, given their critical role in 
establishing and analyzing the effects of admission policies. Resources of this nature will allow for a discussion 
of research and practice at various types of institutions. 
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2. Uneven Preparation for Tests 

Understand test preparation and take into account disparities among students with 
differential access to information about admission testing and preparation 

In the past decade, test preparation courses have proliferated and raised a number of issues. 

Some research has been done to determine the extent to which coaching can increase scores, 
but it is not definitive. An objective assessment of the effects of various coaching methods 
on scores is clearly needed. Are the numbers of coached students sufficient for there to 
have been a discernable effect on aggregate scores? ... How does the content of coaching 
programs relate to the subject domains the tests are designed to measure? Coaching also 
raises questions about fairness: such programs generally cost hundreds or thousands of 
dollars, and while some public school districts offered coaching to their students, it is clearly 
not equally available to all. (Myths & Tradeoffs, 1999) 

Since the Myths and Tradeoffs report was published in 1999, there has been exponential growth in the test 
preparation industry. Based on available data, the test preparation industry generates more than $1 billion 
annually. 1 The industry would not, of course, exist without two primary inputs: (1) a majority of colleges and 
universities require standardized admission tests for admission, and (2) families sustain the demand for such 
services with disposable income and a desire to give their students ‘a leg up' in the admission process 

As long as colleges and universities require standardized tests for admission, admission test preparation activities 
will thrive. The precise increase in scores as a result of test preparation for an individual is difficult to determine. 
However, reliable academic research suggests that an overall point increase of between 20 and 30 points on 
the SAT appears to be typical for test preparation activities. (Briggs, 2001) This 20-30 point average gain is 
considerably less than the 100 point or more claims that test preparation companies often make for their short 
term programs. Although no reliable and independent research exists supporting the veracity of such large 
gains, they have been accepted as “conventional wisdom” promoted by test preparation companies and often 
not questioned by the media. The Commission believes NACAC can help ensure that students and families are 
aware of the findings of objective researchers on the subject. 

As yet, there is insufficient research to assess fully the effect of the many different types of test preparation 
on standardized admission test scores. While short-term programs do not appear to be particularly effective 
(especially given their cost), less is known about long-term test preparation over many months or even years. 

We do know, however, that test preparation is likely to provide an increase — however nominal — in standardized 
admission test scores. Access to test preparation will always be differentiated based on family income, school 
setting, and other variables external to the student. As a result, students without the financial resources to gain 
access to test preparation may, in effect, be penalized for lower test scores in some admission and scholarship 
scenarios, especially in those that use test scores in a ‘cut score' or non-contextualized linear fashion. NACAC 
research indicates that many students, particularly those from modest economic backgrounds, attend high 



Estimate based on available reports to the Securities and Exchange Commission for test preparation companies 
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schools with large student-to-counselor ratios or have no college counselor at all. (Hawkins, 2007) Such students 
have very few sources of advice about test preparation. Moreover, the ability of more affluent students to afford 
both test preparation and multiple test administrations, particularly when they are able to report the highest 
scores to colleges and universities, puts students of more modest economic means at further risk of being 
overlooked in the transition process. 

The Value of Test Preparation 

After a decades-long debate about whether standardized admission tests can be “coached,” college admission 
counseling professionals, test preparation companies, and the test agencies are all in agreement that students’ 
scores can be improved at least to some extent by participation in activities designed to prepare students for 
the tests. The lingering question remains how much impact such activities — which range from taking practice 
exams to intense, expensive, one-on-one tutoring — have on the test score. According to a white paper on 
standardized admission testing commissioned by NACAC: 

During the last 15 years, several well-designed research studies have produced consistent results 
about the magnitude of score improvement that results from SAT coaching. Becker (1990), Powers 
and Rock (1999), and Briggs (2001; 2004) all concluded that the average gain from SAT coaching 
is between 6 and 8 points on the verbal section and between 14 and 18 points on the math section. 
Coaching studies on tests other than the SAT are quite scarce. Research suggests that coaching 
produces small benefits on the ACT (Briggs, 2001; Scholes & McCoy, 1998). (See Zwick, 2002, 
Chapter 7, and Kaplan, 2005, for additional reviews of coaching research.) Although many testing 
companies long maintained the position that test preparation programs were largely ineffective, the 
sponsors of all major admissions tests now produce test preparation materials, seemingly a tacit 
acknowledgment that preparation can be beneficial. (Zwick, 2007) 

To ensure that the Commission’s work is informed by the most current research, NACAC has commissioned a 
new, more comprehensive white paper on test preparation that will be available in early 2009. The Commission 
hopes that the white paper commissioned by NACAC will help to further research that will keep the public fully 
informed about test preparation. The paper is intended to: 

1. ) Describe and summarize ‘definitions’ of test prep programs. 

2. ) Synthesize and summarize existing academic research on the effects of admission test preparation. 

3. ) Advise the association on a message to the public about conclusions of academic research on the 

effectiveness of test preparation for admission testing. 

4. ) Advise the association on future research needs in this area. 

5. ) Provide recommendations to admission officers and high school counselors for implementing policies 

and training on how to account for the effects of test preparation in the college admission process. 

The Commission concurs with the Myths and Tradeoffs report, which suggests that “[a]n objective assessment 
of the effects of various coaching methods on scores is clearly needed." 
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Commission Recommendation: Test Preparation Research 

The Commission recommends that NACAC pursue relationships with academic researchers and foundations 
that may support an extended “objective assessment" of the effects of test coaching methods to provide current, 
unbiased information to colleges and universities. The Commission calls on stakeholders from all sectors of the 
standardized testing community — including test vendors and test preparation companies — to commit resources 
necessary to establishing an ongoing research program at NACAC to conduct periodic research into the effects 
of various methods of test preparation. 



Ultimately, the Commission believes this effort will accomplish two important goals. First, pooling resources into 
an objective source for admission information mitigates against self-interest in research conducted by any single 
actor. Second, the establishment of an ongoing research effort allows for the development of best practices that 
may provide greater knowledge for stakeholders seeking information about effective test preparation methods 
on a small or large scale. State and federal governments might help to verify research on the value of test 
preparation given the vast sum of money spent by the public on these activities. 



Commission Recommendation: Building the Base of Research 

The Commission also calls on high schools and colleges to share their own institutional research on test 
preparation to fully develop a knowledge center on test preparation. Such a knowledge center would contribute 
to a greater understanding of the vast array of methods of test preparation, and would enable individual schools 
to benefit from shared perspectives with other schools. 



Fairness of Admission Test Coaching 

A key point of concern about standardized admission testing is the ability of privileged students to obtain test 
preparation that may provide an advantage over their less privileged peers. While substantial inequities exist 
in the entirety of the K-12 educational system, colleges and universities — for the most part — are responsible 
for the requirement that students take admission tests. Therefore, the Commission believes that colleges and 
universities have a unique responsibility to mitigate the inequitable effects of test preparation. 

According to the previous NACAC white paper on standardized testing: 

The effectiveness and ethics of commercial test preparation for admission tests, particularly the SAT, 
have long been the subject of controversy. Currently, the coaching debate tends to focus on the 
question of whether coaching, because it is likely to be most accessible to those who have already 
benefited from a lifetime of educational advantages, presents an impediment to test fairness for poor 
and minority test-takers. Powers and Rock (1999) and Briggs (2001) found that coached SAT-takers 
came from more affluent families than uncoached candidates. . .Although average coaching effects are 
apparently quite small, it is legitimate to question the fairness of a system in which some test-takers 
can afford coachins and others cannot. It is clear that coaching programs are here to stay, and that 
it is impractical, if not impossible, to create admissions tests that are not susceptible to coaching. 
Minimizing the impact of coaching on test fairness, then, requires that the availability of free and low- 
cost test preparation be increased. (Zwick, 2007) (emphasis added) 
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Fairness of College Admission Policy and Practice 

The NACAC Statement of Principles of Good Practice require that colleges “not use minimum test scores as 
the sole criterion for admission, advising or for the awarding of financial aid." The Commission believes that 
this requirement is well-known and widely accepted among colleges and universities. The Commission further 
believes, as will be noted in a subsequent chapter, that some colleges, college-related scholarship organizations, 
and corporations are not as diligent as they could be in the application of this principle. 

For most colleges and universities, admission test scores are rarely (if ever) used as a uniform “line in the sand” 
beyond which no student may enter the institution. However, there are a substantial number of institutions that 
rely on an academic index, which averages or otherwise compiles numeric academic achievement indicators 
into a single indexed number. Preliminary decisions about whether a student is definitely admissible, possibly 
admissible, or not admissible may be made on the basis of the index score. 



Commission Recommendation: Considerations for Admission Index Formulas 

In such systems, the Commission recommends that admission policymakers and practitioners remain aware 
of the implications of inequitable access to test preparation as they design and implement index systems. 
Maintaining a commitment to contextual decision-making in admission — ensuring that there is enough flexibility 
to evaluate candidates based on the entirety of their application — is critical, in the opinion of the Commission, 
to the fairness of such systems 



Adequate Provision of Test Preparation 

In addition to micro-level participation in test preparation, macro-level actors increasingly administer test 
preparation activities to large numbers of students. According to preliminary results from NACAC’s 2008 
Counseling Trends Survey, approximately 70 percent of high schools offer test preparation resources of some 
type to their students. Also, the advent of state administration of standardized admission tests for all students — 
either for college preparation or statewide assessment for accountability purposes — has precipitated district- or 
state-level participation in test preparation programs. 

Due to the lack of alignment with K-12 subject matter, preparation for standardized admission tests in the high 
school classroom detracts from the most important element of a student’s college preparation — understanding 
core subject matter. Grades in college preparatory courses are the single most important factor in the 
undergraduate admission decision and the best predictor of first year college grades. 



Commission Recommendation: Comprehensive College Preparation 

Secondary schools seeking to provide test preparation would be best advised to offer it as part of a continuum of 
college preparatory activities that includes other informational ‘coursework,’ such as information about financial 
aid eligibility, familiarization with the college application, overview of the college application and admission 
timeline, essay writing assistance, parent/family involvement, and other services. 



Best Practices for Test Preparation 

Commission members agreed that the best form of test preparation is focused on core knowledge content and 
on skills that will help prepare students for their academic future. Members of the Commission agree that, due 
to the inevitability of test preparation, there are aspects of test preparation that may be considered best practice. 
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Defining test preparation takes on added urgency in instances where preparation for standardized tests infringes 
on the ability of teachers to spend adequate time on core curriculum. Students, families, counselors, schools, or 
school districts seeking appropriate test preparation for standardized admission tests should require that each 
of the following attributes are present in the test preparation program: 

• Familiarity with test question format 

• Familiarity with test administration procedures 

• Alignment with skills necessary to master college preparatory coursework 

• Instruction in basic study habits and skills 



Commission Recommendation: Collecting Promising Test Preparation Practice 

The Commission recommends that high schools and other organizations submit research to NACAC with the 
purpose of establishing a trusted source for best practice and professional development. From this central 
knowledge base, college admission counseling professionals, supported by the association, can continually 
assess and share information about test preparation that is aligned with the development of student study skills, 
academic achievement, and test familiarization. 




NACAC Testing Commission Report, September 2008 



3. Test Misuse 

Draw attention to possible misuses of admission test scores 

While appropriate uses for standardized tests are generally well-established, the Commission identified a num- 
ber of well-known instances of apparent test misuse that persist. Such examples include: 

• The use of test score cut off points as a sole screening factor for awarding scholarship money 

• The use of score data to evaluate institutional “quality" (national ranking systems and bond ratings) 

• The use of admission test scores as evaluative measures of a high school student’s achievement in 
their coursework or of secondary school performance 

This list is not all-encompassing. Rather, it constitutes a starting point for those test misuses that the Commission 
believes are most prevalent and necessary to address. 

The 1999 Myths and Tradeoffs report asked the question, “[T]o what extent are test scores being used in inap- 
propriate ways? Is it possible to obtain this information, given both institutions’ reluctance to reveal the details 
of their practice, as well as practical constraints on the collection of detailed, comprehensive data?” The report 
also poses the questions, “What measures could improve the understanding of the benefits and limitations of 
admissions tests and reduce the risk of misuse of test scores? What steps could test makers, accreditation as- 
sociations, higher education associations, and others take to ensure that sound test use policies are developed 
and followed?" 

Specific Examples of Test Misuse 

National Merit Scholarship Program 

The Commission believes that the time has come to end the practice of using “cutscores," or minimum admis- 
sion test scores, for merit aid eligibility. The scarcity of aid and the advantages affluent students have in gaining 
access to preparation for admission tests or pre-tests require us to demand change. 

The use of cutscores for admission has long been inconsistent with ethical admission decision-making, and in 
2004 the NACAC Assembly adopted a restriction on the use of cutscores for the awarding of financial aid. 2 Or- 
ganizations such as the National Center for Fair and Open Testing (FairTest) have long advocated for change in 
this area, and this Commission believes that the time to have a public debate on this issue is overdue. 



Commission Recommendation: Cease Using Cutscores As Initial Scholarship Eligibility 
Screen 

Accordingly, the Commission calls upon the National Merit Scholarship Corporation to cease using non-contex- 
tualized PSAT scores as the “initial screen” for eligibility for National Merit Scholar recognition or scholarships. 3 



2 Addition to NACAC Statement of Principles of Good Practice, adopted by NACAC Assembly, 2004. 

3 Information about National Merit eligibility and selection processes are available at http://www.nationalmerit.org 
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While we commend the National Merit Scholarship Corporation for providing financial assistance for post- 
secondary education, the Commission believes that among the misuses of admission tests or pre-tests, the 
National Merit Scholarship competition contributes to the misperception of test scores as sole measures of 
“merit” in a pervasive and highly visible manner. As stated by the Office of the President of the University of 
California, “usingthe PSAT exam alone to eliminate the vast majority oftest takers from National Merit Scholarship 
consideration is inconsistent with the principles that standardized tests should be used in conjunction with other 
factors in measuring merit and that major decisions should not be made on the basis of small differences in test 
scores." 4 

Moreover, the Commission believes that problems associated with the use of cutscores as an initial screen are 
compounded by the establishment of different cutoff points for different states. Additionally, the Commission 
is concerned that the effects of such a preliminary screen may result in the unfair exclusion of subgroups of 
test takers. Research conducted for the University of California system illustrates the challenge in determining 
whether inequities exist, as “neither the College Board nor the NMSP [National Merit Scholars Program] has 
been willing to disclose the percent of National Merit Scholars who are Black, Hispanic, or American Indian. Nor 
have they been willing to disclose the percentage of poor students or non-native speakers of English who win 
National Merit Scholarships... nor... any information regarding the background of students who become National 
Merit Scholars.” (Hayashi, 2005) 

The Commission holds no illusion about the scope of the project. We are aware that the National Merit Scholarship 
is a private endeavor, an exclusive partnership with a testing agency, explicit in its scope and purpose, and not 
a universal benefit for all students. Nevertheless, the Commission believes it is critical that the National Merit 
Scholarship Corporation cease the practice of using a PSAT cutscore as the initial screen, and move toward a 
practice that could be condoned by current standards for test score use. 

The Commission also asks that institutions that commit institutional resources towards funding National Merit 
Finalists and Semi-Finalists reconsider whether or not their commitments are founded upon solid and equitable 
principles given the current selection processes. The Commission also calls on the College Board to explain why 
it allow the use of its PSAT as a cutscore, a practice that is clearly at odds with best practices in the use of test 
scores. 

Evaluations of Institutional Quality 

U.S. News and World Report Rankings 

SAT and ACT scores for first-year students at colleges and universities comprise part of the rankings compiled 
annually by U.S. News and World Report. The formula for the rankings includes a “student selectivity” score 
that constitutes 15 percent of a college’s overall rank. SAT or ACT test scores of enrollees comprise 50 percent 
of the student selectivity score. 5 



4 Six UC Campuses to Redirect National Merit Funding to Other Merit-Based Scholarships,” Office of the President of the University of California, 
press release dated July 13, 2005, retrieved from http://www.universityofcalifornia.edu/news/2005/jull3.html. 

5 The methodology for U.S. News rankings is available at http://www.usnews.com/articles/education/best-colleges/2008/08/21/about-the- 
rankingsmethodology.html 
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The Commission believes that SAT and ACT scores are not a valid measure of institutional quality. The Commission 
acknowledges that in the minds of the public, institutional quality and selectivity have long been inextricable. 
However, the Commission believes that the continued use of admission test scores as a ranking-related measure 
creates pressure on admission offices to pursue increasingly high test scores. 

The College Board's “Guidelines on the Uses of College Board Test Scores and Related Data” suggest: 

[Educators, the media, and others should] not rank or rate teachers, educational institutions, 
districts or states solely on aggregate scores derived from tests that are intended primarily as 
a measure of individual students. Do not use aggregate scores as the single measure to rank 
or rate teachers, educational institutions, districts, or states. 

The ACT similarly provides guidelines that discourage institutional comparisons: 

[We communicate results of ACT-owned assessments in ways that promote valid interpretation 
and use. To accomplish this goal, we] discourage comparisons of data among schools, districts 
and states, because the data may not be representative of all students in a school, district, or 
state or of the achievement of examinees from one year to the next. Such comparisons may 
lead to invalid conclusions about differences in quality of education. 



Commission Recommendation: End Use of Admission Test Scores as Measures of 
Institutional Quality 

The Commission believes that, as tests designed to provide information about individuals to colleges and 
universities, the SAT and ACT were never designed as measures of the quality of an institution of higher 
education. Accordingly, the Commission encourages U.S. News to eliminate test scores as a measure of 
institutional quality. 



College and University Bond Ratings 

The Commission also considers the use of SAT and ACT scores in bond ratings of postsecondary institutions 
as an inappropriate use of standardized admission tests. Criteria reports for the major bond ratings companies 
indicate that SAT and ACT scores are used to help assess demand using student quality indicators. The bond 
rating agencies contend that higher SAT and ACT scores afford colleges and universities the ability to adjust 
admission standards downward to offset decreased demand. In other words, colleges whose students have high 
admission test scores can survive economic hard times by lowering their admission requirements and accepting 
students with lower test scores. The logic suggests that colleges with lower test scores are less able to ensure 
their financial well-being by adjusting admission requirements. 

Bond ratings firms suggest that “for schools that accommodate students who have not performed well historically 
on standardized tests, [we place] less emphasis on these ratios in [our] analysis and instead focus on the mission 
of the university or college and its success in achieving its recruitment goals." 6 However, the Commission 



“Rating Guidelines for Public Colleges and Universities," Fitch Ratings, July 2004; “Rating Guidelines for Private Colleges and Universities," 
Fitch Ratings, July 2006; “Public Finance Criteria,” Standard & Poor's 2007. 
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believes this statement is not sufficient to mitigate the effect of improper use of admission test scores as a 
measure of institutional financial stability. 

The Commission takes issue with bond ratings that use standardized admission tests for two primary reasons. 
First, the Standards for Educational and Psychological Testing, as well as the guidelines for use of the SAT and 
ACT, maintain clear restrictions against using admission test scores for purposes other than those for which 
the test is designed. Neither SAT nor ACT has ever claimed that admission test scores are reliable indicators of 
institutional financial health. While there is undoubtedly a correlation between institutional financial resources 
and test scores, the Commission believes that the use of such scores confuses correlation with causation, and 
as such, comprises an inappropriate use of test scores. 

Second, the Commission believes that the continued use of admission test scores as bond rating criteria create 
inappropriate pressure on admission offices to pursue increasingly high test scores. While financial ratings firms 
may suggest that higher scores indicate more flexibility in times of scarcity of student applicants, there is little 
evidence to suggest that institutions lower their test score expectations in such instances. To the extent that 
colleges feel compelled to respond to the significance of test scores in rankings and bond ratings, they will tend 
to place greater emphasis on test scores, exacerbating the challenge of appropriate test use in admission. 



Commission Recommendation: Cease Use of Admission Test Scores as Measures of 
Institutional Financial Health 

For these reasons, the Commission urges NACAC to work with college and university leaders, counterpart 
organizations, and bond rating companies to end the use of admission test scores in bond rating evaluations of 
colleges and universities. 



State Assessments 

The Commission accepts that offering students the opportunity to take the SAT or ACT as part of the high school 
course of study may be an effective college access lever, as it may encourage students who did not believe 
themselves to be college-qualified to realize that they are eligible for admission to a wide range of colleges. 

However, the Commission is concerned that the use of the SAT or ACT without significant modification to measure 
student achievement in high schools is an inappropriate use of these tests. Indeed, according to the Standards 
for Educational and Psychological Testing: 

Admission tests, whether they are intended to measure achievement or ability, are not directly 
linked to a particular instructional curriculum and, therefore, are not appropriate for detecting 
changes in middle school or high school performance. (American Educational Research 
Association, 1999) 



Commission Recommendation: Refrain from Using Admission Tests as Measures of 
Achievement for Accountability Purposes 

The Commission recommends that states refrain from using standardized admission tests, without significant 
modification, as evaluators of student achievement, particularly when high stakes accountability measures are 
attached. Admission tests were not designed for this purpose, and are not sufficiently tailored to measure 
progress toward explicit measures for learning in a given state. 
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There is a substantial body of research that can help states, school districts, and schools design measurements 
of “college readiness.’’ In addition, while admission tests are useful to many college and university admission 
offices, they are less useful than a student’s performance in challenging coursework. Therefore, efforts to 
ensure that schools are preparing their students for college should not look to a single admission exam for an 
assessment of their progress, but to the strength of the school’s curriculum and students’ performance in those 
courses as their primary indicator. 

Usage Audit 

The testing agencies, including the College Board and ACT, all publish and promote guidelines for appropriate 
use of their testing products. As an additional measure, the Commission believes that a periodic usage audit 
may be necessary and desirable to protect against misuse of these organizations’ testing products. There is 
precedent in the College Board’s recently-initiated audit of Advanced Placement programs at high schools 
across the United States. 



Commission Recommendation: Institute Admission Test Usage Audit 

The Commission believes that a similar audit by the College Board and ACT may help limit misuse of tests by 
bringing the two organizations’ combined weight to bear on those institutions and organizations that misuse 
admission test scores. 



Commission Recommendation: Abide by NACAC’s Statements of Principles of Good 
Practice and Incorporate Additional Provisions on Test Practice 

In addition to the Standards for Educational and Psychological Testing, the Commission recommends that 
all colleges and universities abide by the standards set by NACAC for ethical and best practice in college 
admission counseling. The following statements are currently contained in NACAC’s Statement of Principles of 
Good Practice (SPGP). The Commission recommends that colleges and universities, as well as NACAC, strive 
to inform staff, administrators, and policymakers involved in admission practice of the importance of adhering 
to these principles. 



Excerpt from NACAC’s Statement of Principles of Good Practice: 

Mandatory Practices and Interpretations 

Members agree that they will: 

• not use minimum test scores as the sole criterion for admission and/or advising; 

• state clearly the requirements for the first-year and transfer admission and enrollment 
processes, including secondary school preparation, standardized testing, financial aid, housing 
and notification deadlines, and refund procedures by 

o stating clearly and precisely the requirements for secondary preparation, admission 
tests and transfer student admission; 

o clearly publicizing policies relating to placement by tests, awarding of credit and other 
policies based on test results. 
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• initially report on all first-year admitted or enrolled students, including special subgroups in the 
reporting of test scores. If data on subgroup populations are also provided, clear explanations of 
who is included in the subgroup population will be made. 

o Postsecondary members will furnish data describing the currently enrolled freshman 
class and will describe in published profiles all members of the enrolling freshman class; 
o Subgroups within the profile may be presented separately because of their unique 
character or special circumstances. 

• work with school officials and other relevant individuals to keep test results confidential as governed 
by law and local regulations; 

• report on all students within a distinct class (e.g., freshman, sophomore, junior, and senior) and 
subgroups, including non-native speakers, in the reporting of standardized test scores. 

Best Practices 

Members agree that they should: 

• educate staff in understanding the concepts of test measurement, test interpretation and test use 
so they make informed admission decisions about the test data; 

• conduct institutional research to inquire into the most effective use of tests for admission 
decisions; 

• refrain from the public reporting of mean and median admission test scores and, instead, report 
scores by the middle 50 percent of the scores of all first-year applicants, admitted and/or enrolled 
students; 

• report test scores for special subgroups that may include athletes or non-native speakers. 
Universities with more than one undergraduate division may report first by division and then by 
special subgroups within divisions. Clear explanations of who is included in the subgroup should 
be made. Those institutions that do not require tests or for which tests are optional will only report 
scores if the institution clearly and emphatically states the limits of the scores being reported; 

• clearly publicize policies, such as placement and awarding of credit, that are based on test 
results; 

• inform students about the tests needed for admission, where students may take them, how to 
interpret the results, and how test results are used for admission; 

• report, in the case of secondary schools, the middle 50 percent of all students tested by discrete 
grade level; 

• work with school officials and other relevant individuals to keep test results in perspective; 
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Recommended Additions to NACAC’s Statement of Principles of Good Practice 

The Commission recommends that the NACAC Admission Practices Committee consider amending the 
Statement of Principles of Good Practice to include the following guidelines for test score use. 

Familiarity with Standards for Educational and Psychological Testing: College admission officials in charge of 
setting admission test policy should be familiar with and adhere to the Standards for Educational and Psychological 
Testing, particularly with respect to test score use and interpretation, test bias, and score differences between 
subgroups. 
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4. Equipping Our Professionals 

Establish opportunities for colleges and universities to educate themselves about the 
appropriate uses of standardized tests. 

In educational settings, those who supervise others in test selection, administration, and 
interpretation should have received education and training in testing necessary to ensure 
familiarity with the evidence for validity and reliability for tests used in the educational setting 
and to be prepared to articulate or to ensure that others articulate a logical explanation of the 
relationships among the tests used, the purposes they serve, and the interpretations of test 
scores. - Standard 13.12, Standards for Educational and Psvcholosical Testine 

Background 

While some school counselors may receive training in standardized testing as part of a counseling graduate 
program, many admission officers regularly review applications and test scores without the benefit of independent 
academic study or training in interpreting standardized test scores. However, there is currently no guarantee that 
any professional — school counselor, college advisor, or admission officer — involved in college admission has had 
independent training in the evaluation of standardized admission test scores. 

As NACAC’s Statement of Counselor Competencies states, it is essential that college 
admissions counselors understand “the proper administration and uses of standardized 
tests and be able to interpret test scores and test-related data to students, parents, edu- 
cators, institutions, agencies, and the public” (NACAC, 2000, p. 11). In order to develop a 
thorough understanding of test use and interpretation, counselors need to have a command 
of the fundamentals of educational measurement and statistics. However, like K-12 school 
personnel, who must also interpret test scores and explain them to multiple audiences, 
admissions counselors may not have had the opportunity to acquire training in the area 
of academic assessment and score interpretation. . . How, then, can we protect against the 
improper use and interpretation of tests and promote their wise and beneficial use? One 
essential step is to provide high-quality training to all test users, including college admission 
counselors. (Zwick, 2007) 

While many professionals may have had some contact with training in graduate education or with the testing 
agencies, the Commission believes it is important to offer education and training to college admission counseling 
professionals on evaluating admission test scores. 



Commission Recommendation: Admission Test Training for Admission Officers and 
School Counselors 

The Commission recommends that NACAC develop a fundamental training program for college admission 
officers, school counselors, and other college admission professionals to ensure best professional practice in the 
consideration and implementation of standardized admission test policies. It is essential that such fundamental 
training be provided by a source other than the testing agencies, whose training is most appropriate for the 
specific use of their own products. 




NACAC Testing Commission Report, September 2008 



The Commission has identified the following components of standardized admission testing knowledge (along 
with the ability to communicate and apply this knowledge) that high school counselors and admission officers 
should have to perform their admission testing responsibilities (across their constituents - students, parents, 
teachers, administrators, general public). Knowledge components are principally arranged by order of logical 
progression (i.e., from before the test to after the test) and acquisition, and each core component is followed by 
examples of individual concepts that comprise each knowledge area. 

1. Institutional Objectives for the Use of Admission Test Results: formal institutional goal(s)for including 
admission tests as a requirement for enrollment, specific role and function of admission testing within 
an institution’s admission criteria, decision points affected by admission tests. 

2. Ethics and Ethical Standards in Admission Testing: basic understanding of Standards for Educational 
& Psychological Testing related to admission testing, NACAC ethical principles, role(s) and limitations 
of admission test information, test uses and misuses, students’ rights throughout the testing and 
reporting process. 

3. Admission Test Properties: components of admission tests (including subtest components), what 
knowledge and skills each test component measures, how the tests measure knowledge and skills, 
test difficulty, how the tests are scored to reflect knowledge and skill attainment, what the test scores 
represent. 

4. Test Preparation: overview of test preparation, preparing for the testing experience, understanding 
the effects of both test familiarization and test preparation on scores (as best understood by academic 
research), ability to understand ramifications of test score increases even of modest size. 

5. Score Disparity: understanding the potential distinction between possible ‘test bias' (including 
gender, race/ethnicity, and SES issues) and valid score disparity, score change and score difference, 
understanding the relationship of K-12 inequity, curricular experiences, and socio-economic situation 
to test score patterns. 

6. Statistical and Interpretive Concepts: margin of error in test scores, scoring index, percentiles, 
norm and comparison groups, admission test benchmarks, test reliability, differences in scores within 
and across admission tests, and other basic statistical concepts related to the understanding and 
interpretation of scores. 

7. Predictive Validity: understanding the meaning of predictive validity - as well as understanding what 
it does not infer, understanding of standard practice in assessing predictive value of tests (and other 
factors), statistical concepts related to validity such as correlation and variance, variations in predictive 
value of tests for different groups, understanding of Freshman GPA vs. success in college (including 
academic performance and student persistence). 

8. Admission Models: understanding of basic operation of admission modelsand howtestsare positioned 
in each model, including (but not limited to) holistic admission, “index" composites, and test-optional 
admission policies. 
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Recommended Additions to NACAC’s Statement of Principles of Good Practice 

The Commission recommends that the NACAC Admission Practices Committee consider amending the 
Statement of Principles of Good Practice to include the following guidelines. 

1. College admission officer training: To assure best admission practice, any college admission officer, or 
other individual acting on behalf of the institution, who is responsible for reading admission applications 
should complete a basic training program in standardized admission tests. Training should be provided 
by an agency or institution that is independent of companies that sponsor the tests or test preparation 
activities. 

2. Counseling professional training: To assure best practice in college admission counseling, any school 
counselor, independent counselor, or other individual engaged in providing professional college 
guidance to students should complete a basic training program in standardized admission tests. 
Training should be provided by an agency or institution that is independent of companies that sponsor 
the tests or test preparation activities. 



NACAC Testing Commission Report, September 2008 



5. Test Score Differences 

Understand differences in test scores among different groups of people, and continually 
assess the use of standardized test scores relative to the broader social goals of higher 
education 

According to the association’s white paper on college admission testing, “ [clollege admissions test results 
often reveal substantial average score differences among ethnic, gender, and socioeconomic groups. In the 
popular press, these differences are often regarded as sufficient evidence that these tests are biased. From 
a psychometric perspective, however, a test's fairness is inextricably tied to its validity.” (Zwick, 2007) The 
Commission believes the question of the effect of test score differences among subgroups on test validity may 
never be conclusively resolved, but can be acknowledged and appropriately factored into admission decisions. 

Score Differences and Disparate Impact 

The Commission is aware that “both sides of the bias argument can offer credible data to support their positions.” 
(Zwick, 2002) The Commission acknowledges that there are distinct score differences between subgroups of 
test takers. A recent study at the University of California succinctly describes the effect of these differences: 

As an admissions criterion, the SAT has a more adverse impact on poor and minority 
applicants than high school grades, class rank, and other measures of academic achievement; 
admissions criteria that emphasize demonstrated achievement over potential ability are more 
likely to promote educational equity. (Geiser, 2007) 

The Commission agrees that by overemphasizing SAT and ACT scores in the admission process, colleges and 
universities may exacerbate existing disparities among under-represented students. While often referred to as 
“common yardsticks” for measuring applicants, test scores (on average) are strongly correlated with student and 
family attributes over which a student has no control (see examples on page 39). 

Questions persist about whether standardized admission tests provide sufficient additional utility to justify their 
use in light of their correlation with wealth, parental education, and race/ethnicity. However, before determining 
whether the existence of a correlation between admission tests and socio-economic or ethnic attributes justifies a 
wholesale de-emphasis or elimination of the current admission tests, the Commission believes that it is important 
to take into account other factors that may contribute to test score differences among these subgroups. Such 
discrepancies, for instance, may have their origins in the inequitable allocation of public education resources, 
an issue beyond the scope of this Commission’s work. 



